The present invention relates to technology of generating a three-dimensional geometric model that is composed of visual information of a three-dimensional body, and more particular to a three-dimensional body photography apparatus for photographing an image of a three-dimensional body having neither a shadow nor shading for employment in generating the three-dimensional geometric model, a three-dimensional geometric model generation apparatus, a three-dimensional geometric model generation method, and a three-dimensional geometric model generation program.
In recent years, computer graphics (Hereinafter, referred to as CG for short) technology has developed remarkably. For this reason, scene images under various situations has become possible to render realistically based on the three-dimensional geometric model that is composed of visual information of the three-dimensional body.
In particular, a human body or a face is one of important objects as an object of rendering, and generation of its precise three-dimensional geometric model has be come of importance. In the conventional three-dimensional geometric model, three-dimensional shape data indicating a three-dimensional shape (three-dimensional coordinates of the surface etc.) of the three-dimensional body to be taken as an object, and visual information such as color and a reflectance at each position on its surface are employed.
If there is the three-dimensional geometric model of the face of a person, it is possible to freely generate a face image having posture, illumination conditions, or expression etc. altered by employing the technology of the CG. As to this technology of the CG, for example, it was described in details in “Open GL programming guide” by Addison-Wesley (4-7952-9710-X).
By these kinds of the CG technology, production of realistic animation video becomes possible. Also, application of producing a simulation system is also possible in which a favorite is selected by changing glasses and a hair style etc. to view it from a free viewpoint, and so forth.
Also, in JP-P2001-283216A entitled “BODY COLLATION METHOD, BODY COLLATION APPARATUS, AND RECORD MEDIUM IN WHICH PROGRAM THEREOF WAS RECORDED.” invented by this inventor, a system was disclosed in which the measured three-dimensional geometric model and color data was employed to generate the face image at the time that the illumination conditions and the posture were altered, and collation was made possible of the face image photographed under various conditions such as a monitoring camera.
By the way, in the event of generating the face image at the time of having altered the posture of a figure, with knowledge of a correspondence relation between each pixel of the face image and each of three-dimensional coordinates (three-dimensional shape data) on the surface of its face, it is enough, and a perspective conversion model is frequently employed as a camera model for describing this.
In the perspective conversion model, in the event that three-dimensional coordinates at each point P of the three-dimensional geometric model of the face were taken as (X, Y, Z), coordinates of the pixel of the image as (u, v), a rotation matrix indicating the posture of the face as R, a position of the face as (Tx, Ty, Tz), and a focal length of the camera as f, the following expression of Eq. 1 is given.
                              [                                                                      x                  ′                                                                                                      y                  ′                                                                                                      z                  ′                                                              ]                =                                            R              ⁡                              [                                                                            x                                                                                                  y                                                                                                  z                                                                      ]                                      +                                          [                                                                                                    T                        x                                                                                                                                                T                        y                                                                                                                                                T                        z                                                                                            ]                            ⁢                                                          [                                                                    u                                                                                        v                                                              ]                                =                                    f                              z                ′                                      ⁡                          [                                                                                          x                      ′                                                                                                                                  y                      ′                                                                                  ]                                                          (        1        )            
By employing Eq. 1 on each point P of the three-dimensional coordinates on the surface of the face, the coordinates of the pixel of its face image are found. And, by painting the above pixel with color of the point P, the face image at the posture R and the position (Tx, Ty, Tz) can be generated.
The CG technology described above allows the face image that takes an optional posture to be rendered, if there are three-dimensional shape data of the face, and color data of the surface. As to the color data for employment at this time, it is general to utilize a brightness value of the image photographed under appropriate illumination conditions as it stands. The above image is called a texture image. Also, a method of employing the brightness value of the above texture image as the color data to render the image is called a texture mapping.
Further, in the event of generating the image at the time of having altered illumination, a reflection model is utilized for describing how brightness on the image of each portion of the three-dimensional body surface alters depending upon a direction of the illumination.
A Lambertian model is a reflection model to be frequently employed for finding the brightness value on the image of each portion of the three-dimensional body that alters depending upon how the illumination is applied. This Lambertian model and the perspective conversion model are normally rigged as a graphics function of a general computer (workstation), and are widely employed.
In the Lambertian model, in the event that a direction of a light source was taken as L→, strength of the light source as |L→|, a vector normal to each point P→ of the three-dimensional body (face) as n→, and a reflectance of the surface as A, a brightness value I of the point P on the image is calculated by the following expression of Eq. 2.
                    I        =                  A          ⁢                                          ⁢                                    n              →                        ·                          L              →                                                          (        2        )            
Not only in the Lambertian model explained here, but also in the other models in which discoloration by pressing etc. was considered, similarly to the expression of Eq. 2, the brightness value of the image is calculated with the three-dimensional shape data (x, y, z), the direction n→ normal to the surface to be obtained from it, and the color data of the surface (and reflectance data) A.
Accordingly, if there are the three-dimensional geometric model of the three-dimensional body, i.e. the three-dimensional shape data, and the color data (and the reflectance data) indicating visual information of the surface, it is possible to generate as the CG the image in the event that the illumination exists in an optional direction L→.
As to a technique of measuring the three-dimensional shape and the surface color of the three-dimensional body, there are various ones. For example, the three-dimensional shape data is measured by analyzing the image photographed by projecting a specific light pattern to the face. Additionally, with regard to the measurement of the three-dimensional shape by a method of projecting the light pattern etc. it is well known to those skilled in the art of the present invention, whereby its explanation is omitted.
In these prior arts, a normal two-dimensional image, to which the pattern was not projected, was photographed as a texture image to the measurement of the three-dimensional shape data simultaneously with, just before, or just after photographing said projected pattern image, and the above texture image was employed as the color data of the three-dimensional body surface for generating the three-dimensional geometric model.
This is because a special measuring apparatus becomes necessary separately for accurately measuring the reflectance, which takes much time. Said texture image is an image in which an influence of a shadow and shading that occurred depending upon how the illumination was applied was added to the reflectance; however this was employed in an approximation-technique manner as a substitute for the reflectance in the prior art.
In the prior arts as mentioned above, the problems as described below existed.
In generating the conventional three-dimensional geometric model, it was difficult to accurately measure the color data and the reflectance data A of each of the three-dimensional coordinates of the three-dimensional body surface to be utilized in the texture mapping, whereby the texture image (two-dimensional image) of the three-dimensional body was employed as a substitute for this.
However, the shadow and the shading occurred to the texture image of the three-dimensional body to be employed as a substitute, whereby the problem existed: the accurate three-dimensional geometric model was impossible to generate.
FIG. 11 and FIG. 12 are views illustrating photography apparatuses 700 and 710 for photographing the texture image for employment in generating the above-mentioned conventional three-dimensional geometric model of the face of the person. FIG. 11 illustrates a technique of utilizing brightness of a room lamp 800 in the location, in which measurement is made without employing a special light source, for photography. FIG. 12 illustrates a technique of illuminating the face with a lamp of a projector 721 for projecting the light pattern to be employed for measuring the three-dimensional shape data for photography.
However, the shadow and the shading occurred to the texture image photographed by such conventional photography apparatuses 700 and 710, whereby the accurate three-dimensional geometric model is impossible to generate. As a result, the image having the posture altered, which was generated by means of the texture mapping, became unattractive.
Also, in the event of utilizing brightness of the room lamp 800 for photography as shown in FIG. 11, the problem exists of being influenced by circumstances of the location in which measurement is made. In the event that the texture image was photographed in such illumination conditions that strong illumination was applied from the right direction of the face, for example, such as the case that a window existed in the right direction of the face and daylight shined, and the case that a wall stood in the left side and plenty of room lamps existed in the right side, to take this as color data for generating the three-dimensional geometric model, the texture image of the face has the shadow that occurred to the left side of a nose as shown in FIG. 10. When such an image that the shadow exists in the left of the nose and the left side of the face is dark is intended to be employed as a texture to generate and view the image from the left side by turning the face, it became an image photographed in the backlight situation, of which visible part was wholly dark.
That is, in the event of desiring to view a clear face image from an optional direction, the texture image needs to be photographed in the situation that the entirety of the face is uniformly illuminated so as to cause the shadow not to occur if circumstances allow.
In the prior art, however, a method was not considered of establishing appropriate illumination conditions for photographing the texture image, whereby the produced CG image became unattractive.
Also, in the prior art, it was not considered to employ the texture image in generating the three-dimensional geometric model as a substitute for said reflectance data A.
It was taken into account to utilize it merely as the color data for display, whereby the appropriate illumination conditions were not established at the time of photographing the texture image. For this reason, the brightness value of the photographed texture image, to which an influence of the shadow and the shading occurred, becomes much different from said reflectance data A. As a result, the texture image was impossible to employ in generating the three-dimensional geometric model in substitution for said reflectance data A.
That is, the light source needs to be arranged at an appropriate position and to be employed so that the shadow and the local shading of the nose do not occur at the time of photographing the texture image. Otherwise, the image generated by means of the CG technology from the three-dimensional geometric model generated by employing the photographed texture image becomes an unimpressive image having the unnatural shadow and shading attached.
For example, in FIG. 11 described above, in the event of employing the image in which the illumination was applied to the face from the left to generate the three-dimensional geometric model by the model of the expression of Eq. 2, it is hardly possible that the shadow intrinsically occurs to the left side of the nose. However, as to the shadowed part of the texture image, its reflectance became a smaller value as compared with intrinsic color. For this reason, such an unnatural image that the shadow existed on the left side even though the illumination was applied from the left was sometimes produced. Also in the event of FIG. 12, the unnatural image in which light was applied from the lower direction became a texture image, whereby the similar problem occurred.
Also, as another example, now think of the case that light was radiated to the face from a front of the face for photography like a photography apparatus 720 of FIG. 13. In this case, the problem of the shadow of the nose becomes small.
In this case, however, as shown in FIG. 9, an angle between a direction normal to the face surface and a direction of the illumination becomes large in the cheek part, whereby the brightness value of the texture image photographed in this part results in being low (dark). Accordingly, even though a forehead part and the cheek part are almost identical in terms of true color (and the reflectance), the problem existed that the cheek part became dark remarkably, and the image generated by means of the CG technology became unnatural.
In such a manner, when the image having the shadow and shading, which was photographed under the non-uniform illumination, is employed (instead of color and reflectance data) to generate the three-dimensional geometric model, the image generated by means of the CG by altering the viewpoint and the illumination conditions looks unimpressive, and becomes an unnatural image.
Thus, the task to be solved by the present invention is to provide a three-dimensional body photography apparatus, a three-dimensional geometric model generation apparatus, a three-dimensional geometric model generation method, and a three-dimensional geometric model generation program that realize generation of the CG that looks impressive by, in the event of generating a three-dimensional geometric model of a rugged three-dimensional body such as the face of the person, photographing the texture image having neither the shadow nor the shading by applying the uniform illumination to the entirety of the surface of its three-dimensional body, and employing this texture image to generate the three-dimensional geometric model.